It seems that I always get a response like "the provided information was not sufficient" compared to the develop branch where it gives an answer in the same conditions.

For example:
PDF file: https://content.iospress.com/articles/data-science/ds190021
Question: "What is hobbit"

It seems that I always get a response like "the provided information was not sufficient" compared to the develop branch where it gives an answer in the same conditions.

For example: PDF file: https://content.iospress.com/articles/data-science/ds190021 Question: "What is hobbit"

Most likely it is not ingested

Most likely it is not ingested

Yes, looks like something with import or ingestion. Tried to run the import with develop branch and then run the RAG pipeline from feature/updater and it worked.

  File "learn2rag/pipeline/app.py", line 125, in <lambda>                                             
    chunks = await loop.run_in_executor(executor, lambda: list(sync_gen()))                                                                               
                                                          ~~~~^^^^^^^^^^^^
  File "learn2rag/pipeline/app.py", line 122, in sync_gen                                        
    for chunk in generate.generate_stream(question, results, opt_config, request_id=request_id):                                                          
                 ~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                          
  File "learn2rag/pipeline/generate.py", line 36, in generate_stream                                  
    context = "\n\n".join([context_template.format(source=result.payload['path'], content=result.payload['content']) for result in search_results]) # type
: ignore[index]                                                                                                                                           
                                                          ~~~~~~~~~~~~~~^^^^^^^^                                                                          
KeyError: 'path'                                                                                                                                          

Example of result:

ScoredPoint(
id='2516e9c5-f34f-4249-8454-f3ba8ed8fc50',
version=5,
score=0.52725434,
payload={
  'content': '...',
  'source': '....pdf',
  'content_hash': '74bc2f7667474af9c82af19bc89a3c1a27b12f48575cef7316d728a5ec50f7d8',
  'chunk_hash': '306b5f122e5e7ee5a2222f8bf37d22d9',
  'title': '...',
  'uri': '',
  'loader_id': '420fe88d-8171-4225-bd55-a740feba85c8',
  'document_id': '',
},
vector=None,
shard_key=None,
order_value=None)

  File "learn2rag/pipeline/app.py", line 125, in <lambda>                                             
    chunks = await loop.run_in_executor(executor, lambda: list(sync_gen()))                                                                               
                                                          ~~~~^^^^^^^^^^^^
  File "learn2rag/pipeline/app.py", line 122, in sync_gen                                        
    for chunk in generate.generate_stream(question, results, opt_config, request_id=request_id):                                                          
                 ~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                          
  File "learn2rag/pipeline/generate.py", line 36, in generate_stream                                  
    context = "\n\n".join([context_template.format(source=result.payload['path'], content=result.payload['content']) for result in search_results]) # type
: ignore[index]                                                                                                                                           
                                                          ~~~~~~~~~~~~~~^^^^^^^^                                                                          
KeyError: 'path'                                                                                                                                          

Example of result:

ScoredPoint(
id='2516e9c5-f34f-4249-8454-f3ba8ed8fc50',
version=5,
score=0.52725434,
payload={
  'content': '...',
  'source': '....pdf',
  'content_hash': '74bc2f7667474af9c82af19bc89a3c1a27b12f48575cef7316d728a5ec50f7d8',
  'chunk_hash': '306b5f122e5e7ee5a2222f8bf37d22d9',
  'title': '...',
  'uri': '',
  'loader_id': '420fe88d-8171-4225-bd55-a740feba85c8',
  'document_id': '',
},
vector=None,
shard_key=None,
order_value=None)

'Path' was Changed to 'source' at some point